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ABSTRACT 


The need for an auxiliary performance measure for the 
design of constrained optimal controls for linear regula- 
tors is shown. Several auxiliary performance measures are 
compared, and the maximum of the absolute degradation over 
the admissible initial states is selected as the auxiliary 
performance measure. Computational algorithms which make 
extensive use of existing library subprograms are developed 
for the design of constrained optimal controls in those 
cases where the control vector is specified as constant or 
piecewise-constant linear feedback of the output vector. 
Numerical examples including a third-order system and a 
time-varying system are given to illustrate the applica- 


tions of the proposed algorithms. 
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I. INTRODUCTION 


The linear regulator is one of the most extensively 
studied and well-known problems of optimal control theory 
[Y, 3egle THe importance of linear regulators comes from 
the fact that many practical control problems can be formu- 
lated in the form of a linear regulator. In addition, the 
feedback form of the optimal solution to the problem is a 
very desirable feature. The practical application of the 
optimal solution to the linear regulator problem suffers 
from two serious drawbacks. One is the time-varying nature 
of the feedback gains, and the other is the need for exact 
physical measurements of the value of the awe vector at 
every instant of time. In general, not the state vector of 
the system, but the output vector of the system is available 
for measurements. It is possible to reconstruct the state 
VeEGEon leSidey or to obtain an approximation to the state vec- 
tor that is best in some sense when noise is present [6] ; 
however, in many cases, these schemes to obtain the state 
vector are not justified due to economic considerations. 

Since the mathematical formulation and solution of the 
optimal linear regulator problem by Kalman [2], the design 
of suboptimal controls, which are easy to implement, yet 
not too inferior to optimal ones has attracted the atten- 
tion of several investigators. 

Schoenberger [7] has given solutions to a class of 


problems in which the initial conditions are known. He has 


also presented a procedure for minimizing the expected 
value of the performance measure when the probability dis- 
tribution of the initial conditionspiseisnowes 

Meditch [8] has proposed an approximate method of de- 
coupling complex systems which simplifies the computation 
of the time-varying feedback gains. 

Koivuniemi [9] has specified the form of the feedback 
control and has determined the unknown parameters to mini- 
mize the maximum degradation of the system performance over 
the admissible initial states. 

Rekasius [10] has proposed a control law that minimizes 
the maximum (with respect to all initial states) relative 
deviation in the value of the performance measure with re- 
spect to the optimal performance measure. Although his 
method avoids some of the difficulties related to minimax 
problems, it is impractical for systems of higher than the 
second order. 

In Kleinman and Athans' [11] formulation of the sub- 
optimal linear regulator problem, easily realizable time 
functions are used to generate the suboptimal time-varying 
feedback gains. The unknown parameters are determined to 
minimize the expected value of the performance measure over 
the admissible initial states, which are assumed to be uni- 
fommly Gistiibuted on eines. bv eissmie rok 

Kleinman, Fortmann and Athans [12] have given an ap- 


plication of Kleinman and Athans' [11] approach. They 





have presented an algorithm for the numerical solution of 
the problem, for, the case where the system is time-invari- 
ant and the suboptimal feedback gains are constrained to be 
piecewise constant. 

Levine and Athans [13] have also used an average per- 
formance measure, but they have let the control vector to 
be a time-varying function of the system output rather than 
the system state. 

Salmon [14] has proposed an algorithm which can be ap- 
plied to the design of controllers for systems with unknown 
parameters, including initial conditions. In this method, 
the controller parameters are found in such a way that the 
maximum of either the system performance measure or the 
so-called performance sensitivity with respect to the un- 
known system parameters iS minimized. The performance sen- 
Sitivity is defined as the percentage or the absolute in- 
crease in the performance measure from that of an ideal con- 
troller caused by the unknown system parameters. The ideal 
controller is defined as the controller which is capable of 
measuring unknown system parameters and generating its own 
parameters accordingly. 

In this thesis, after a brief review of the theory of 
eptamal control of linear regulators, the need for an aux- 
iliary performance measure for the formulation of the con- 
strained optimal control problem is shown and several aux- 
iliary performance measures are compared using a simple 


second-order example. Then the constrained optimal control 


problem is formulated and a numerical method of solution 
for the case of constant feedback gains is given. Next, an 
approximate, but very practical, solution to the problem 
for the case of piecewise-constant feedback gains is pro- 
posed. Numerical examples that illustrate the properties 


of the techniques are given. 
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Il. OPTIMAL CONTROL OF LINEAR REGULATORS 


Consider the linear, time-varying system 


x(t) 


Ae Ge), + BiMe)matt) (1) 


Vaksts) Git) sale) (2) 


where x(t) is the n-dimensional state vector, u(t) is the 
m-dimensional unconstrained control vector, 57 (eae is the 
r-dimensional output vector, and A(t), B(t) and C(t) are 
time-varying matrices of dimensions n x n, n xm, andrxn, 


respectively. Determine the control fe sy that minimizes 


the performance measure 


Jix.,tul-)) = 5 x7 (t)Hx(t,) 


ys 


me | [x (t)Q(t)x(t) + w(t) R(t) u(t) Jat (3) 
C 
O 


where to and te > to are the initial and final times, re- 
spectively, x = x(t.) is the initial state, H is a real 
symmetric positive semi-definite matrix, Q(t) is a time- 
varying real symmetric positive semi-definite matrix, R(t) 
is a time-varying real symmetric positive definite matrix, 


and the final state x(t ,) is free. 


i 


The solution to this problem is well-known [1,2,3,4]. 


The optimal control exists and is given by 


u*(t) = - R(t) B2 (t)K(t) x(t) 


Sea P* (ex (t) (4) 


where K(t) is the unique symmetric positive definite solu- 
tion of a matrix differential equation of the Riccati type 


given by 
K(t) = - K(t)A(t) - A (t)K(t) - Q(t) 
+ K(t)B(t)R™”(t)B’ (t)K(t) (5) 
with the boundary condition 
=H. (6) 
The state of the optimal system is the solution of 
x(t) = [A(t) - B(t)E*(t) x(t) (7) 


with the boundary condition x(t.) = xX. The optimal per- 


formance measure iS given by 


J (x oaks piasli=\).) 
oe as) = ee) 


Kaiti) Ke (8) 
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The matrix K(t) has another interesting property. For ar- 


bitwary € <t, and x(t) 


fe 


I* (x(t), t) = 5 x" (t)K(t)x(t) (9) 


Some comments on the results just given are in order: 
first, the optimal control is a linear, time-varying feed- 
back of the system states. (See Fig. 1). Second, the 
matrix Riccati differential equation, Eq. (5), is a set of 
* coupled, first-order, nonlinear differential equations 


which can be solved numerically beginning at t = t, and in- 


£ 
tegrating backward in time to t = ty using the boundary con- 
dition (6). Actually, only n(n+l)/2 of these equations 

must be solved because K(t) is symmetric. 

In implementing the controller block in Fig. 1, the 
most appealing scheme is to calculate K(t,) by integrating 
(5) backward off-line and then to calculate T(t) on-line 
by integrating (5) in the forward direction using previous- 
ly calculated K(t.) However, this is not possible in the 
time-varying case because of the instability of (5) in the 
forward time direction [2]. Even if the system is time- 
invariant, on-line integration is not always desirable due 
to increased complexity ofe the system. Another possibility 
is to provide storage for the elements of the feedback gain 
matrix F*(t) and to store the precalculated values. This 
1s possible because the elements of ie E@) do not depend on 


the system state but depend only on time; therefore, 


3 


estimate 
of x(t) 





a priori knowledge of the initial state is not racy ETI 
However, this scheme suffers from the same drawback as the 
previous scheme, namely, increased complexity. 

Suppose that F*(t) is readily available. Even then 
the realization of the optimal control, u*(t), requires com- 
plete knowledge of the system state x(t). Reconstructing or 
estimating the state vector [5,6] from the knowledge of the 
output, y(t), may add addi etonenl complexity to the control- 
ler. 

In many instances it is possible to decrease the com- 
plexity of a control system considerably by constraining 


the controller configuration and allowing a small increase 
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in the system performance measure with respect to optimal. 
If the adjustable parameters of such a controller are de- 
termined in some optimal fashion, then the control signal 
is called a constrained optimal control. In the following 
sections the design of constrained optimal controls for 
systems which can be modeled in the form of linear regula- 


tors will be discussed, 


dis 


III. FORMULATION OF THE CONSTRAINED 


OPTIMAL CONTROL PROBLEM 


For the system and the performance measure given by 


(ry), (2)8ard (3) ébhsiger) the’ controlelaw 
u(t) = - P(t)y(t) (10) 


where P(t) is anm x r matrix which will be called the out- 
put-feedback gain matrix. The physical meaning of this con- 
trol is as follows: the control signal is generated from 
the system output vector rather than from the system state 
vector as in the optimal control. Substituting (2) in (10), 


one obtains 


We) 


- P(t)C(t)x(t) 


lib 


SNe) ec (e) Ci) 


The value of the performance measure for the control law 


(ll) is defined as 


I> 


Tc, EE ))) = Gee es CoD (12) 


u(-) = -P(-)y(-) 
Tf F(t) defined by (11) is not equal to Bee) defined by 


(4) for all BS eS then the control (10) is not op- 


fF! 


timal. In general, 


T(x, t Bed!) =e ae, Em) qe) 


16 





It is shown in the Appendix that 


T 
D4 
a @ | 


hole 


T(x 5,t,.P(+)) = V(t .P(-))x, . (14) 


where ey is the symmetric, positive definite solution of 


the matrix differential equation 
V(t) = - v(t) CA(t) - B(t)P(t)c(t) J 
-[A(t) - B(t)P(t)e(t)]" v(t) 
~ Q(t) - C7 (t) P(t) R(t) P(t)C(t) (15) 


with the boundary condition 
v(t.) =H (16) 


From the point of view of implementation, the control (10) 
has distinct advantages over the optimal control: first, 
knowledge of the system state vector x(t) is not needed. 
Second, constraints may be imposed on P(t) to make the 
structure of the control simple, e.g., P(t) may be required 


to be constant or piecewise constant. 


A. NEED FOR AN AUXILIARY PERFORMANCE MEASURE 
It is reasonable to suggest that the elements of P(t) 


be chosen so that the performance measure (12) is minimized, 


lL 


Note that V(t) depends on the values that P(t) attains in 
the interval between t and Gee. “However this will be shown 
explicitly only when t = t as in Eq. (14). 


iL) 


and at the same time the constraints imposed on the elements 
of P(t) are satisfied. However, there is one major diffi- 
culty in proceeding in this fashion: it is the need for a 
priori knowledge of the initial state of the system. It 

was pointed out in the previous section that although 

J™ (x,t) is a function of the initial state, the optimal 
feedback gain matrix F*(t) is independent of the initial 
State. However, this is not the case with the control (10), 
in general. Not only T(x ,t ,P(-)), but also the best out- 
put-—feedback gain matrix, P(t), depends on the initial state. 
If the initial state is known a priori, then it is possible 
to find optimal values for the elements of P(t), for 

—E st st 


Oo fe 


that may have been imposed on them, by minimizing the per- 


satisfying constraints of whatever complexity 


formance measure at least numerically, if not analytically. 
If the initial state is not known, then one approach is to 
define an auxiliary performance measure which is independent 
of the initial state. Some concepts that will be needed in 
defining an appropriate auxiliary performance measure will 
be given first. 

The Absolute Degradation, denoted by AD, will be 


defined as 
ry i _ +k 
AD(x.,t .P( )) = I(x ,t Bl )) J (x 7 t)) ; (17) 


that is, the absolute degradation is the increase in the 
performance measure with respect to the optimal value caused 


by application of a control given by (10). 
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The Relative Degradation, denoted by RD, is given by 


Hitiart AP ()) - J” (Kat ® 
RD(x pt ,P(-)) < ———_—— i = = (18) 
—oyete’ 
Both the absolute and relative degradations are always non- 
negative quantities. 

Assuming that the possible initial states may lie 
within a hyperspherical region, with center at the origin 
and having a specified radius, there is no loss of general- 
ity in further assuming that the set of admissible initial 
states, denoted by X, consists of all points which lie .on 


the unit hypersphere. This is because, for any admissible 


x. and any p 20 , 


J(px i,t vul-)) = 5 Pax Kiba), 
= p* d(x ,t .ul-)) , (19) 


1.€., the performance measure due to an initial state which 
is not on the unit hypersphere is a constant times the per- 
‘formance measure due to the corresponding point on the 

unit hypersphere, 

There are two basic approaches in choosing an auxiliary 
performance measure: one is the statistical approach, the 
other is the worst initial state approach. In the first ap- 
proach, it is assumed that the probability distribution of 
eae Miitss! seate rs known, and a statistical function Of 


the performance measure that is independent of the initial 


nD 


state is taken as the auxiliary performance measure. Several 
authors [7,11,12,13] use the expected value of the pertomie 
ance measure over the admissible initial states as the aux- 
iliary performance measure. Normally, a simple probability 
density function, such as a uniform distribution is assumed. 
This makes the expected value of the performance measure a 
very simple function of the V matrix found from (15). 
Several variations on this approach depending upon the con- 
straints imposed on the structure of the control and the 
assumed admissible initial states are given in the references 
cited above. 


In the worst initial state approach [9,10,14], the max- 





imum (worst) over the admissible initial states of any one 
of three basic quantities - performance measure, absolute 
degradation, and relative degradation - can be taken as the 
auxiliary performance measure. Therefore, there are four 
functionals of P(-) which can be used as the auxiliary per- 


formance measure. These are: 


J, (P(-)) ee ee (20) 
—e 

Ty (B(-)) = Max J(x,/E,-B(+)) (21) 
O 

Ss oe” (22) 
=e) 

J,(P(+)) —e RD(x.,t).P(-)) (23) 
=O 


yAG) 





The output-feedback gain matrix, which minimizes the 
auxiliary performance measure is called the constrained op- 
timal output—feedback gain matrix and is denoted by Pi(t). 
The constrained optimal control, denoted by u(t), is given 
by (10) with P(t) replaced by B.(t). 

The performance measure resulting from the application 
of a constrained optimal control is called the constrained 


optimal performance measure and is denoted by J (x0, t))- 


B. A COMPARISON OF SEVERAL AUXILIARY PERFORMANCE MEASURES 
1. Example 1 
The purpose of this example is to illustrate the 
differences that result from the use of various auxiliary 
performance measures. The method of obtaining the solutions 
will not be discussed, but rather the implications of each 
different result will be indicated. 


The system to be controlled is described by 


x4 (t) O 1 | x, WE) 0 

= + u(t) (24) 
ze olf) 0 0 | x5 (t) 1 ' 
y, () ik O x, (€) 

= B (25) 
y,(t) 0 DW alps ('€) 


The performance measure for optimal control is 


2 
Nee sue yy"= J bey?(e) - ele wale : (26) 


22 


The constrained optimal control is to be in the form 


y, (t) 
ee Pi ae 
Yo (t) 
x, (t) 
= - /P ia Pia , (27) 
x5 (t) 


where Pil and Pj are constants that minimize the auxiliary 
performance measure. 

This problem was solved four times, using the four dif- 
ferent auxiliary performance measures given by (20) - (23). 
The values obtained for Pil and Pi> for each case are tabu- 
lated in Table 1. The conclusion that can be drawn looking 
at that table is very important: The values obtained for 
the parameters of the constrained optimal control depend 
very much on the auxiliary performance measure used. 
Therefore, the proper choice of an auxiliary performance 
measure 1S very significant. The values that the several 
performance quantities attain asefunctiens of the imitiall 
state are also different in each case. Since the system is 
only second order and the initial state is assumed to lie 
on the unit circle, just one quantity, namely the ratio of 


the two components of the initial state vector, x(t )Ay,(t.), 
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Auxiliary Per- 
formance Measure 


Table 1. Constrained optimal feedback 
gains for different auxiliary 
performance measures used in 
meee 1. 











E (T(x, tr P) } 


Mm €X 
—=@ 







Mex J(x_,t_,P) 
x €X =9 © 






Max AD(x_,t_,P) 
=O =n 

x EX 

—O 





is enough to specify a particular initial eee. The op- 
timal performance measure J*, constrained optimal perform- 
ance measure Je: absolute degradation AD, and relative de- 
gradation RD are plotted as functions of x, (t.) /x, (E,) on 

one graph for each of the auxiliary performance measures in 


Pros. 2, 3) 4 eamd 5. 


1 

It does not make any difference whether x7(t,) and xj(t)) 
eee beth in the figstequadrant or both in the third querd- 
rant (Similarly for the second and fourth quadrants) due to 
symmetry. 


a3 








[ie 


Zs 


ee - ee re 


eT 





= bd 


(ee > A 


Various performance values as 
functions of initial state. 
(Maximum relative degradation 
minimized, ) 


2 x,/x,> 4 


Various performance values as 
functions of Smitialestate. 
(Expected value of performance 
measure minimized.) 
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ee 
ae a aE. 
= ai pee r Lael a 


-4, ie 0 ee ae 


Fig. 4. Various performance values as 
Functions of initial state. 
(Maximum performance measure 
minimized, ) 


— eS ee 


Fig. 5. Various performance values as 
haNncHenS VOL initial state, 
(Maximum absolute degradation 
minimized, ) 


For this example, i€ is clear that using (the maxamem 
relative degradation as the auxiliary performance measure 
(Fig. 1) is not very desirable. Although the maximum RD 
is less than 0.2 it amounts to a large absolute degradation 
for initial states where the optimal performance measure is 
already high. 

When it is assumed that the initial states are uniform- 
ly distributed on the unit hypersphere and the expected 
value of the performance measure is used as the auxiliary 
performance measure (Fig. 3) the relative degradation goes 
as high as 0.43. But this occurs for initial states where 
the optimal performance measure is quite low, and the ab- 
solute degradation does not amount to a large value. In 
this case the J. curve is closer to the J* curve on the 
average than the other three cases. But, unless separately 
calculated as in this example, nothing can be said about 
maximum absolute degradation which may be more than allow- 
able. 

Fig. 4 shows the case when the maximum J is used as 
the auxiliary performance measure. In this case the con- 
strained optimal control is almost as good as the optimal 
control when the constrained optimal performance measure is 
the maximum. In other words, the control is the best 
(closest to the optimal) when the initial condition is the 
"worst". This is a very desirable feature. However, it 


results in a very pessimistic design, i.e., the absolute 
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degradation for the other initial states becomes quite 
high. As far as the absolute degradation is concerned, al- 
lowing a small increase for the worst initial state may re- 
sult in a significant improvement for the other initial 
states, as is the case in Fig. 5. 

In the last case where the maximum absolute degrada- 
tion is used as the auxiliary performance measure (Fig. 5), 
the constrained optimal performance measure as a function 
of the initial state is made as close to the optimal per- 
formance measure as possible by minimiZing the maximum dif- 
ference between the two, i.e., the maximum absolute degra-~ 
dation. When this approach is used one is assured that the 
system performance will not be degraded more than a certain 
amount with respect to the optimal system performance no 
matter what the initial state is. 

2. Some Comments on Example 1 

Example 1 has demonstrated that the choice of an ap- 
propriate auxiliary performance measure is a significant 
step in the design of constrained optimal controls for 
linear regulators. It is easy to reject the maximum rela- 
tive degradation as the auxiliary performance measure in 
almost all applications. After all, one would not want to 
get farther away from the optimal performance as the con- 
trol task becomes more demanding. The remaining three de- 
finitions for the auxiliary performance measure are all 
reasonable and each one of them can be used depending on 


the application. During the remainder of this work, the 
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maximum absolute degradation will be used as the auxiliary 
performance measure. The results obtained can be easily 
applied to the case where the maximum performance measure 
is used as the auxiliary performance measure. The case 
Where the expected value of the performance measure is used 
as the auxiliary performance measure is easier to handle 
mathematically (especially when the probability distribu- 
tion of the initial state is assumed to be uniform on the 


unit hypersphere) and is treated elsewhere [7,11,12] . 


Cis STATEMENT OF THE CONSTRAINED OPTIMAL CONTROL PROBLEM 
It is assumed that the system is described by the state 


and output equations 
x(t) = A(t)x(t) + B(t)u(t) ee(1) 
y(t) = C(t) x(t) (2) 


given previously. The performance measure for optimal con- 


broly as 


J(x ,t ,u(-)) = s Stellen x(t) 


i: 


£ 
a 1 | [x (e)Q(t)x(t) + u"(t)R(t)u(t) Jat . (3) 


, Uu 
fe) 
Let the set of admissible initial states be X, and let X 


consist of the points which lie on the unit hypersphere. 


Let 8% denote a specified class of real time functions. 
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The constrained optimal control problem is to find a 


control 
u_(t) = - P_(t)y(t) (28) 


such that the elements of P.(¢t) are members of © and the 


auxiliary performance measure 


Ta(B(-)) = Max AD(x,,t,+2(+)) (29) 
“oe 


is minimized. 
The value of the auxiliary performance measure corre- 


sponding to P_(t) is denoted by a” p Laees, 


J, = J,(P(-)) 


ee) = Pals) 


Min Ji(P(-)) . (30) 
P(-)eQ 


The magnitude of a 1S an indication of how close the con- 
strained optimal control is to the optimal control. How- 
ever, a normalized number is usually more meaningful in 
making comparisons; therefore, the Normalized Degradation 
of the system, denoted by ND, will be defined as 
Ts 
DS (32) 


* 

Max J (x ot.) 
xX EX : 

= 


29 


ND is zero if the performance of the constrained optimal 


control is the same as the performance of the optimal con- 
trol for all initial states; otherwise, it will be a posi- 


tive number. 


30 





iV. SOLUTION.OF THE CONSTRAGNED 


OPTIMAL CONTROL PROBLEM 


A. CONSTANT OUTPUT-FEEDBACK GAIN MATRIX 

In this section, the constrained optimal control prob- 
lem will be solved for the case where the elements of P(t) 
are constrained to be constant in the interval between t. 
and tC. - 

1. Development of the Method 


It is desired that the constant matrix P. be found 


such that 
+» ae 
J, (P,) ae 
= Min Max AD(xX.,t +P) . (32) 
P x_€X 
= 


This is a minimax problem, and there are many difficulties 
associated with the solution of such problems: first, in 

general, an iterative numerical procedure in which a com- 

plete maximization process is required for each minimiza- 

tion step is necessary [9]; second, no matter how smooth 

the function AD(x.,P) is in both an and P, the function 


J_(P) = Max AD(x,,t 
x €X 


oP) (33) 


is not, in general, differentiable on the elements of P. 
tin partreullar, it miay"not be differentiable in Pat the 


point yielding the minimum [14,15]. Therefore, some 


ilk 


method which does not require the partial derivatives of 
J_ AP) with respect to the elements of P is more suitable 
than, for example, a method in which the gradient vector is 
needed and the partial derivatives are evaluated by making 
small perturbations. 

It is possible to simplify the procedure consider- 


ably by noting that 


a _ «~* 
AD(x.,t .P) = T(x. t +P) J (x,t) 


J 
N|r 


T 
Ke Vee Kt) 1x 


Aly 
x 
ao) 


I> 


No |e 


W(E pee (34) 


1s a real quadratic form in xX. and that the following 





well-known [16,17] theorem about the extremal properties 
of the eigenvalues of a real quadratic form is applicable. 
Theorem: The global maximum of a real quadratic 
form on the unit hypersphere is equal to the largest eigen- 
value of the quadratic form, and moreover the corresponding 
eigenvector is the vector drawn from the origin to the 
point on the hypersphere where the quadratic form achieves 
its maximum. Therefore 
Max AD(x_,t.,P) = + A, (P) (35) 
ee =O" Ole 2° lee 
=6 
where A, (2) is the largest eigenvalue of W(t. ,P) defined | 


by (34), and the problem is to minimize d, (2) with respect 


BZ 


to the clements of P ; that is, to find the constant matr2x 


les such that 


rs 
Je.) oe 


Min ,(P) . (36) 
Pp 


i 
2 


Such a formulation has several advantages over the formula- 
En given by (32). First, 


dh, (PB) = 2 Max AD(x.,t +P) (25) 


x €X 
=o 


can be evaluated without the need to find the point where 
the maximum occurs. This is very important because (37) is 
evaluated many times during the minimization process, and 
any Savings in the number of necessary computations is sig- 
nificant. Second, the programming required is extremely 
Simple because very efficient methods are available for 
finding the largest eigenvalue of a real symmetric matrix, 
and many computer facilities have several standard subrou- 
tines that can be used for this purpose. However, it should 
be noted that a subroutine that finds all eigenvalues (and 
eigenvectors) of a general square matrix is not as suitable 
for this purpose as one which finds only the largest eigen- 


value of a real symmetric matrix. 
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2. Algorithm for Computing the Constant Output- i 

Feedback Gain Matrix - Algorithm I 

a. Calculate K(t)) by integrating (5) backward 
with the boundary condition (6). 

b. Guess startimgyvalues, for the entries of By 

c. Find P that minimizes X, (2) by uSing a sub- 
routine that preferably does not require the evaluation of 
the gradient. Such a subroutine will, in general, require 
the evaluation of d, (PB) many times. This can be accomplish- 
ed as follows: 

(1) Calculate V(t..P) by integrating (15) back- 
ward with the boundary condition (16), using the current 
VauMe ole. 

(2) Calculate W(t..P) using (34). 

(3) Find the largest eigenvalue, A, (PB), of 
W(t .P) using a Suitable subroutine. 

gd. If the location of the point where the worst 
degradation occurs is desired, then find the eigenvector 
corresponding to A, (B.)- 

In applying this algorithm to the examples given 
in the rest of this thesis, the minimizations were accom- 
plished using a diibPetitine- that performs the pattern 


search method of Hooke and Jeeves [18,19]; the largest 


1 
Subroutine DIRECT, NPS Computer Facility. 
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eigenvalues were found using a subroutine- that utilizes 
Givens-Householder method; and integrations were performed 
using a Runge-Kutta-Gill fourth-order method-. 
3. Application of Algorithm I - Example 2 

The purpose of this example is to illustrate the 
application of the described algorithm to a simple second- 
order system, and also to illustrate the importance of using 
the appropriate algorithm. 


The system to be controlled is described by 


x, (t) 0 1 x, (t) 0 
; = + u(t) (38) 
X(t) -3 -1 x5 (t) 1 
y, (t) 1 O x, (t) 

2 , (22) 
Yo (t) O i Xo (t) 


The performance measure for optimal control is 


Sap 


at ae 2 2 2 
T(x ,ule)) = 5 | Fae (t) + x, (e) + uv (t)ddt. (44) 
O 


The constrained optimal control is to be in the form 
y, (t) 


see I (41) 
Us ili) Pia | y5(¢) 


1 
Subroutine GIVHO, see Ralston and Wilf [20]. 


2 
Function RKLDEQ, NPS Computer Facility. 
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where Pai and Pj are constants. #Expanding, the Rieeaiea 


equation (5) with 


one obtains 
Ok a ko, (t) 
Eopiitlee 1s oe) - WeaAE) + 3k, (t) +k), (Ele (t) a 
Koote) Fe alkene ety) = it ae) 


with the boundary conditions 


~ ~ 


Ki, (3). = Bp ee so, GAD" 05 (43) 


Note that due to the symmetry of K(t) only three equations 


are needed. Expanding the equation (15) with 


1 O 
P= [Py] Pyo| and c= 
O il. 
one obtains 
2 


V1, (¢t) = 2(3+p 14) Vz (#) -l- Pay 


(44) 


e 


_ 2 
Vo9 (t) = 2(1+p, 5) Vo5(t) -2V, 5 (t) -l-pj5 
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with boundary conditions 


Walter demetnn ae) mwa, (  ) = 0. (45) 


Again due to symmetry only three equations are needed. 
The proposed algorithm gives the solution for the 


constant output-(or state) feedback gain matrix as 
Eee tomes? (078097 | 


The normalized degradation (ND) is 0.00055 which shows that 
the constrained optimal.control is almost as good as the 
Optinial cemtrol. 

Koivuniemi [9] treats the same system as an ex- 
ample for the design of a constrained optimal controller 
which he calls a specific optimal controller. He uses the 
same auxiliary performance measure as here, but obtains a 
result which is quite different. He gives the constant out- 


put-(or state) feedback gain matrix as 


P_ = [0.69 0.76] 


Assuming that the initial states lie on the unit circle, 

the optimal performance measure con constrained optimal 
performance measure Jas absolute degradation AD, and rela- 
tive degradation RD are plotted as functions of Xo (t,) /x(t,) 
using the P. matrix obtained here and the PB. matrix given 
in ([9] in Pie. mand 7 respectively, for comparison. 


Clearly the result found here results in a much better 


a7 





Oy hae 


Fig. 6. Various performance values as 
functions of initial state - using 
feedback gains found in Example 2. 





Fig. 7. Various performance values as 
functions of initial state - using 
feedback gains given in Ref. 9. 
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performance for all initial states than the result given by 
Koivuniemi. This is also obvious from a comparison of the 
normalized degradation (ND) figures. It is 0.057 for 
Koivuniemi's design -- much higher than 0.00055 obtained 
here. 

This example has illustrated the importance of 
using an efficient algorithm for the solution of the mini- 
max problem. Apparently, the algorithm given in [9] has 
failed at some point, resulting in a solution which is not 
as good as it could be for this example. 


If the output of the same system (38) is given by 


x, (t) 
y,(t) = (elena (46) 
x. (t) 


and the constrained optimal control is specified in the 


form 
u. (t) = — Py ¥ ft) . (47) 
then the expansion of (15) with 
2 enc ll eon 
gives 
ee 5a ati). mee 
lll dal Peer aig 
Vio (t)= Vio ft) - V1 (Ct) + (34+P)4) Voo CE) (48) 


Vo5(t)= 2V55(t) - 2V45(t) - 1 


ie 


with boundary conditions (45). Applying the same algorithn, 


one obtains 


P. = pj, = -0.236 


which gives a normalized degradation (ND) of 0.137. 


Kolvunvenr’s resulteftér this case is 


P. =e -0.25 





which agrees reasonably well with the result found here. 


Evidently, the algorithm given in [9] works in this case. 


Bus PIECEWISE-CONSTANT OUTPUT-FEEDBACK GAIN MATRIX 
In this section, the constrained optimal control prob- 
lem will be solved for the case where the elements of P(t) 
are constrained to be piecewise-constant in the interval 
between t_ andt.,.. 
O ig 
1. Development of the Method 





Suppose that there are N subintervals of time, each 
of duration At, during which the elements of P(t) are con- 


stant. Let 
t. = ty + iAdt, iam , lee... ees (49) 


and let 
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Since PB. is anm x r matrix and since there are N such con- 
stant matrices, one can minimize the auxiliary performance 
measure given by (29) with respect to the mrN elements which 
are to be determined and obtain the piecewise-constant P(t). 
However if mrN is a large number, the minimization of the 
auxiliary performance measure with respect to such a large 
number of variables may be a very difficult and time-con- 
suming task. 

A much simpler, yet possibly more powerful and 
practical approach will be used here. This approach is in- 
spired by the Principle of Optimality of Bellman [1]. Sup- 


pose that the time t has been reached and there remains 


N-1 
only one interval during which control can be applied. At 
Chie peintys LE x(ty_4) were known a priori, then to deter- 
mine Pye would be a simple parameter optimization problem, 


and Puy would be determined to minimize J(x(ty 4). 


n-1'Py-" 
However, x(t,_,) is not known a priori, therefore, one is 
again confronted with the problem of defining an auxiliary 
performance measure. Fortunately, the solution proposed 

for the constant-P case in the previous section can be ap- 
plied to this case also by simply taking ty as the ini- 
tial time and applying the Algorithm I. This gives the 
elements of P. , N-1)° 


This solution for the last interval is equivalent to assum- 


as well as K(t a V(t, _4) and W(t 


N-1l 


ing that the admissible x(t, _4) lies on the unit hyper- 
sphere (without loss of generality) and minimizing the max- 


imum absolute degradation with respect to the admissible 


4l 


x(t)_4)- Similarly, Pvp can be determined using ty? and 
ty-1 2s the initial and the final times respectively and 
using K(ty_4) and V(t, _4) obtained as a result of the pre- 
vious application as the boundary conditions in performing 
the integrations. The same procedure is applied to all 


Subintervals until Poe K(t,) and V(t.) are obtained at the 


last application. 


2. Algorithm for Computing the Piecewise-Constant 


Output-Feedback Gain Matrix - Algorithm II 


a. Let i=N,. 
ee eee een! 2 ee 
c. Apply Algorithm I to obtain Pely K(t,_,) and 


ign LT adie le, emo. 

a 2) ee ee ie Bs fc) ee Me om 

Geile Or: 

3. Some Comments Concerning the Application of Algo- 

pei Sonim AhAL 

The optimal value for the auxiliary performance 
measure, J” , obtained by applying Algorithm II will be 
greater than the value that would be obtained if the mini- 
mization were accomplished at one step with respect to mrN 
elements as indicated earlier. However, it should be re- 
membered that in the latter case the "worst" initial state 
(the initial state that causes the maximum absolute degrada- 


tion) is assumed at time ty only and not at the beginning of 
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each subinterval, whereas, in the method of solution pro- 
posed here, the constant»output-feedback gain matrix for 
each subinterval is determined to minimize the maximum ab- 
solute degradation at the beginning of the subinterval. 

It 1S even possible that the normalized degradation 
may increase with a small increase in the number of subin- 
tervals. This is because the "worst" is assumed to occur 
more often and the parameter values that are chosen with 
this in mind may result in higher normalized degradation 
than the case where "worst" is assumed less often or just at 
the beginning. However, when this occurs, it still does not 
mean that increasing the number of subintervals is not ad- 
vantageous. This entirely depends on the application. If 
the system to be controlled is in on Se: reanene which is 
known very well and no disturbances are anticipated, then 
increasing the number of subintervals in the manner proposed 
here does not result ina "better" system, unless there is 
a decrease in normalized degradation. If piecewise-censtant 
output—feedback gains are desired in such a case, then one 
Should perform the minimization at one step with respect to 
all mrN elements as mentioned earlier. However, in practice 
many systems operate in environments where unpredictable 
disturbances can occur at any time. In those cases, increas- 
ing the number of subintervals even when the normalized de- 
gradation does not decrease is desirable, at least from a 


conservative design point of view. 


43 


When all of the states of a system are available 
as outputs and the number of subintervals approaches infin- 
ity, i.e., the constraints are relaxed, then one expects 
the constrained optimal control to approach the optimal con- 
trol and the normalized degradation to approach zero. This 
was the case in all the examples studied by the author. 

4. Application of Algorithm II 
a. Example 3 
The purpose of this example is to illustrate 

the application of Algorithm II to a third-order system. 


The system to be controlled is described by 





| x, (t) A Ox (Oia: 
x(t) | = Op AOmocwe eae | CUTE) (51) 
X(t) PO Of ez (CE a a 
y, (t) lego 0 | x(t) | 
Pea ee oho | (Sau (52) 
| y(t) | a Oe 1 x, (t) | 


The performance measure for optimal control is 


5 dol AG x, (t) 


I(x ,u(+)) = fe Lac jaGiedipes lite dor alts )ed -1 1 0 x, (t) +u-(t) dt. 


Qe Oseald x, (t) (53) 


di 

This system is treated in Ref. 12 where constant and 
piecewise-constant gains are computed using a different 
auxiliary performance measure. 
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The constrained optimal control is to be in the form 


y, (te) 
fase ieeep, 4 (t) peed) vst) | (54) 


y(t) 


where [p,, () Py (t) Ba allel is a piecewise-constant out- 
put-feedback gain matrix. The matrices needed for the ap- 


plication of Algorithms I and II are: 


-3 0 O 2 a o af 
Ae #0 0.2 eS) 2 co | ls 
#9 =-2 0 | -1) 7 oF 7 
y) ne 
Q=j;-2 2 0 R= 2 BP = (p,,(t) P,5(t) p(t)! 


© 
© 
© 


and H = 0. This problem was solved for the one-subinterval 
(a constant feedback gain matrix) case using Algorithm I, 
and for two and ten-subinterval (piecewise-constant feed- 
back gain matrices) cases using Algorithm II. The elements 


BT Kit) are plotted in Figs. 


i a gh lg a a 
8, 9, and 10 for each case. The normalized degradation is 
0.0356 for the one-subinterval case and decreases to the 
values 0.0135 and 0.0025 for the two and ten-subinterval 
cases, respectively. As the number of subintervals is 


further increased, the constrained optimal feedback gains 


become closer to the optimal feedback gains, and the 


45 


on 


+ 
’ 
ee ee er ee eee 





—0.4. 











o_o 


Optimal and constrained optimal 
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Fig. 9. Optimal and constrained optimal 
feedback gains for Example 3 -- 
Two subintervals. 


AT 





; 
+ O02 ree Ethene eH ereean 


—1.0 


Optimal and constrained optimal 


gains for Example 3 -- 


Ten subintervals. 
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normalized degradation decreases. When the number of sub- 
intervals is fifty the normalized degradation is 0.0002. 
Clearly, for this example, one can get as close to the op- 
timal as desired by increasing the number of subintervals. 
b. Example 4 

The purpose of this example is to investigate 
and compare the effectiveness of using different states of 
a linear regulator to form the feedback control, and to 
provide another illustration of the applications of Algo- 
rithms I and II. The system to be controlled is again 
given by (51) and the performance measure for the optimal 


control by (53). This time the output is a scalar given by 


x, (t) 
y(t) = ¢ | x, (t) (31) 


x(t) 


where C is a lx 3 matrix, with only one element equal to 
One and the rest equal to zero. In other words, only one 
of the states is available as an output. The constrained 


optimal control is specified in the form 
u(t) = - p(t)y(t) (56) 


where p(t) is the piecewise-constant output-feedback gain. 
Table 2 gives the normalized degradations for different 
outputs and for various numbers of subintervals. Clearly, 


c= [0 0 1] results in the best constrained optimal 


AS 


Sonecrol, 1.6%, x, (t) is quite effective in forming the con- 
trol. Considering that the normalized degradation is 0.5653 
without any control (u(t)= 0) it can be seen that c=[2100] 
or Cc = [0 1 0] provide very little improvement over no con- 
trol. When c = [0 0 1], there is a small increase in the 
normalized degradation when the number of subintervals is 
increased from one to two. This was mentioned as a possi- 
bility previously. However, further increase in the number 


of subintervals decreases the normalized degradation again. 

















One Two Four 
Subinterval Subintervals Subintervals 


0.5022 0.5022 0.5022 
Ue wale. | 0.5252 0.4767 0.4419 
lo ome 0.0907 0.0942 OmoOeee 


“Table 2. Normalized degradations for different outputs 
and number of subintervals for Example 4. 








c,. Example 5 
In this example, Algorithms I and II are ap- 
plied to a second-order, time-varying system. The system 


is described by 


20 


x Pah ii "| 








x, (€) eee ast | | *1' i 

; ; u(t) (57) 
Xo (t) 0 O Xo (t) | 0.08167 
y, (t) JL 0 x, (t) 

= ; (58) 


y(t) 0 i x (t) 


The performance measure for optimal control is 


10 
1 


s(x su(e)) =4 [C200 x,7(t) + 2u2(t) Jae. (59) 
O 


The constrained optimal control is to be in the form 


y, (t) 


Gacy — —lp,, (ey pi 5 tt)! (60) 
y. (t) 


where [p,, (e) on, tee is a piecewise-constant output-feed- 
back gain matrix. Expanding the equations (5) and (15) 


with 


a 

These equations represent the perturbed motion of the 
flight of an airplane in a vertical plane on a gliding path 
inside the equisignal zone of the glide radio beacon, with 
constant velocity. For more detail on the description of 
the system, see Example 1 of Ref. 21 where an optimal con- 
troller is designed for the system. 
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Ia Sse 0 
ae LS 

0 0 | 0.08167 

ik O 200 O 
ge g 4 

0 1 | 0 0 


R = 2 and H = 0 and applying Algorithms I and II, the prob- 
lem was solved for several agrees with various numbers of 
subintervals. The elements of F(t) = P.(t)c and 

F*(t) = RB K(t) are plotted in Figs. 11 and 12 for one 
and two-subinterval cases respectively. The normalized de- 
gradation is 0.02 for the one-subinterval case and 0.0852 
for the two-subinterval case. This increase in the normal- 
ized degradation with an increase in the number of subin- 
tervals was mentioned previously as a possibility in the 
comments following the description of Algorithm II. How- 
ever, as the number of subintervals is increased further, 
the normalized degradation decreases and approaches zero, 
e.g., it is 0.0205 for the ten-subinterval case and 0.0015 
for the fifty-subinterval case. This example shows that 
the application of Algorithms I and II to time-varying sys- 
tems is as straightforward as the application to time- 


invariant systems. 


2. 
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Optimal and constrained optimal feedback 


gains for Example 5 -- One subinterval. 
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Fig. 12. Optimal and constrained optimal feedback 
gains for Example 5 -- Two subintervals. 
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V. CONCLUSIONS 


A. SUMMARY 

The problem of designing optimal controls for linear 
regulators under complexity constraints has been investi- 
gated. The need for an auxiliary performance measure was 
demonstrated and several candidates for the auxiliary per- 
formance measure were compared. The maximum of the abso- 
lute degradation over the admissible initial states was se- 
lected as the auxiliary performance measure. A method was 
developed and a computational algorithm (Algorithm I) was 
given to solve the resulting minimax problem for the case 
Where the feedback gains are constrained to be constant. 
A conservative engineering approach inspired by the Princi- 
ple of Optimality was used for the solution of the same 
problem for the case where the feedback gains are constrain- 
ed to be piecewise-constant. It was assumed that the "worst" 
initial state could occur at the beginning of each subin- 
terval and the parameters were determined accordingly. A 
computational algorithm (Algorithm II) was given for this 
purpose. 

Both algorithms were programmed in FORTRAN IV for the 
IBM 360 in use at the NPS Computer Facility. Since the 
algorithms rely heavily upon existing, well-established 
library subprograms, the programming required was very 


simple, yet efficient and reliable. In none of the problems 
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solved did the results depend on the initial guess for the 
parameter values. For the minimization step of the algo- 
rithms a gradient scheme in which partial derivatives were 
evaluated by making small perturbations was also tried and 
gave the same results as the pattern search in all cases. 
The gradient algorithm, however, required using more than 
one initial guess in some instances, and did not improve 
the computation time observed with the pattern search 
method. 

Algorithm II does not add much complexity to Algorithm 
I, nor does it increase the computation time. This is be- 
cause most of the computation time is used for integration, 
and when the number of subintervals is increased, there is 
a corresponding decrease in the number of integration steps 
per subinterval and hence no significant change in computa- 


tion time occurs. 


Die PROBLEMS FOR FURTHER STUDY 

It would be worthwhile to investigate the application 
of the given algorithms to linear tracking problems because 
these problems have wide applications and methods for easi- 
ly realizable controllers for such systems are very desir- 
able. 

Another worthwhile extension is the application of the 
given methods to the design of simple compensators for 
automatic control systems. For example, for a third-order 


linear regulator with only one output, a second-order 
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compensator could be combined with the system and then in 
addition to the output feedback gain, the compensator para- 
meters could be optimized in a minimax sense for the over- 


all fifth-order system. 


a 


APPENDIX 


DERIVATION OF EQUATIONS (14),(15) AND (16) 


For the system and the performance measure given by 
(1), (2) and (3), consider the control law given by (10) or 


(ji). Substieacing (1a ings onexsobtaams 
x(t) = [A(t) - B(t) F(t) Ix(t). (Alas 


Let (t,t) be the transition matrix [1] of (A.1), then 


the motion of (A.1) is given by 


x(t) = @ (t,t .)x (A.2) 


and @ (t,t) satisfies the differential equation 


d = 
scedeaee) = TA e) eB ( eye (tc) oe) (A. 3) 
with the boundary condition @ (tot) = tin addveron, 


the final state is related to any preceding state by 


ate (UE ec (A.4) 
and, in particular, to the initial state by 
x(t,~) O(t.,t x - (A.5) 


@ (t.,t) seems” the differential equation 


t This derivation follows the outline given in [8]. 


: This can be shown by differentiating (A.4) with respect 
to t and noting that dx(t-¢)/dt = 0. 
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d 
ae P(t t) = -O.(t,, t) [A(t) -B(t) F(t) J (A.6) 


for all t, and t, with the boundary condition @ (t,t 
Stlbetatutking (11) ame 3)je0one obtains 


Ks REPEC) S x" (t,)H x(t.) 
t | 
1 ii T 
7 1 | x (t)[Q(t)+F° (t) R(t) F(t) ]x(t)dt. (A.7) 
c 
O 


Further substitution for x(t) and x(t¥,) using (A.2) and 
(A.5) gives 

3(x_,t..P(-)) =2x ee 

=O ln Ya a) ies £*"s6 


e 


+ 
hole 


S* (te )Q(t) +E” (t) R(t) F(t) 1G (tle )x dt. 


= 
O |*% Fh 
3 


(A.8) 


Combining the two terms gives 


_ ge 
T(x ,t.B(-)) =F xo { oT (et HO (tert) 


e 


F 
4 | Br(e,b.) (Q(t) +F" (t) R(t) F(t) J@ (t,t at } x : 
O 


{A.9) 
Defining 


4 


v(t) £6 *(t,,t)H ® (t,t) 


E 


ie 
+ | &* (7, t) (Q(T) +E" (T)R(T)E(T) ]® (7, t) ar, (A.10) 
t 


Se, 


then V(t. ,P(+))> is the term in the braces in (A.9) and 


it tae, (inh) a, (le) 


which is repeated here for convenience, is obtained. 
To show that (15) and (16) hold, Ace tale LS {(A.10) witen 


respect to t, with the result 


V(t) = tH & (t_,t eS (t_.t)H Oty, ¢) 


b 
= 
ee 
7 | 1 & 7 (7, t) (Q(T) +E" (7) R(T) E(T) 1. (7, t) 

t 


+ & 7 (7, ¢) (Q(T)+E" (T)R(TIE(T) 1& (7, t)} ar 


- fote)+r  (e)R(E)F(t)) . (Aen 


SUbsereuting (A.G) for @ (t,t) and @.(T,¢t) (noting the 


necessary correspondence between t. and T in the latter 


f 


case) gives 


r See the footnote on page l7. 


6 


£ 
‘ tient) = | £F,t)d™ then by Beabnamez's rule 
t 


E 
1 
cieCe O iterate) 
dt t Ot 
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W(t) = - (A(t) - Blt) E(t)" (t,,t)H & (t,,€) 


Tt 


= A iter, thee eNeae) - B(Ey ECE) 


t 
fi 
- | (A(t) -B(t) F(t) ]°&2 (7,) (Q(T) +E" (7) R(T) E(T) 1& (7,t)ar 
c 


t 
f 
a | 1 (7,t) [o(7) +F1 (7) R(T) F(T) 16 (r,t) LA(t) -B(t) F(t)Jar 
e t -—S —sS 


- g(t) - Fo(t)R(t) F(t) . (A.12) 


Combining the first and third terms and similarly the second 
and fourth terms of (A.12), and identifying V(t) as given 


by (A.10) one obtains 
w(t) = - (A(t) - B(t)F(t)]" v(t) 
- v(t) (A(t) - B(t) F(t) ] 
- o(t) - F(t) R(t) F(t) (A.13) 


(15) is obtained by substituting F(t) = P(t)C(t) in (A.13); 


and (16) is obtained by letting t = t, in (A.10). Taking 


fe 
the transpose of both sides of (A.10) does not change the 
right side, hence V(t) is symmetric. That a! is positive 
definite can be seen by substituting (14) and (8) ‘in (13) 


and noting that K(t)) is positive definite. 
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